Penalty Records was started in 1995 as a joint venture between Tom Silverman (from Tommy Boy Records) and Neil Levine. Silverman acquired Levine's shares in 1999 and Penalty Records was absorbed by Tommy Boy Records. A new version of Penalty Records was relaunched in 2003 through Rykodisc and is still operating, although now through EMI.